Virtual Interpersonal Touch 1 Virtual Interpersonal Touch: Expressing and Recognizing Emotions through Haptic Devices

The current work examines the phenomenon of Virtual Interpersonal Touch (VIT), people touching one another via force-feedback haptic devices. As collaborative virtual environments become utilized more effectively, it is only natural that interactants will have the ability to touch one another. In the current work, we used relatively basic devices to begin to explore the expression of emotion through VIT. In Experiment 1, participants utilized a 2 DOF force-feedback joystick to express seven emotions. We examined various dimensions of the forces generated and subjective ratings of the difficulty of expressing those emotions. In Experiment 2, a separate group of participants attempted to recognize the recordings of emotions generated in Experiment 1. In Experiment 3, pairs of participants attempted to communicate the seven emotions using physical handshakes. Results indicated that humans were above chance when recognizing emotions via VIT, but not as accurate as people expressing emotions through non-mediated handshakes. We discuss a theoretical framework for understanding emotions expressed through touch as well as the implications of the current findings for the utilization of VIT in human computer interaction. Virtual Interpersonal Touch 3 Virtual Interpersonal Touch: Expressing and Recognizing Emotions through Haptic Devices There are many reasons to support the development of collaborative virtual environments (Lanier, 2001). One major criticism of collaborative virtual environments, however, is that they do not provide emotional warmth and nonverbal intimacy (Mehrabian, 1967; Sproull & Kiesler, 1986). In the current work, we empirically explore the augmentation of collaborative virtual environments with simple networked haptic devices to allow for the transmission of emotion through virtual interpersonal touch (VIT). EMOTION IN SOCIAL INTERACTION Interpersonal communication is largely non-verbal (Argyle, 1988), and one of the primary purposes of nonverbal behavior is to communicate subtleties of emotional states between individuals. Clearly, if social interaction mediated by virtual reality and other digital communication systems is to be successful, it will be necessary to allow for a full range of emotional expressions via a number of communication channels. In face-to-face communication, we express emotion primarily through facial expressions, voice, and through touch. While emotion is also communicated through other nonverbal gestures such as posture and hand signals (Cassell & Thorisson, in press; Collier, 1985), in the current review we focus on emotions transmitted via face, voice and touch. In a review of the emotion literature, Ortony and Turner (1990) discuss the concept of basic emotions. These fundamental emotions (e.g., fear) are the building blocks of other more complex emotions (e.g., jealousy). Furthermore, many people argue that these emotions are innate and universal across cultures (Plutchik, 2001). In terms of defining the set of basic emotions, previous work has provided very disparate sets of such emotions. Virtual Interpersonal Touch 4 For example, Watson (1930) has limited his list to “hardwired” emotions such as fear, love, and rage. On the other hand, Ekman & Friesen (1975) have limited their list to those discernable through facial movements such as anger, disgust, fear, joy, sadness, and surprise. The psychophysiology literature adds to our understanding of emotions by suggesting a fundamental biphasic model (Bradley, 2000). In other words, emotions can be thought of as variations on two axes hedonic valence and intensity. Pleasurable emotions have high hedonic valences, while negative emotions have low hedonic valences. This line of research suggests that while emotions may appear complex, much of the variation may nonetheless be mapped onto a two-dimensional scale. This notion also dovetails with research in embodied cognition that has shown that human language is spatially organized (Richardson, Spivey, Edelman, & Naples, 2001). For example, certain words are judged to be more “horizontal” while other words are judged to be more “vertical”. In the current work, we were not concerned predominantly with what constitutes a basic or universal emotion. Instead, we attempted to identify emotions that could be transmitted through virtual touch, and provide an initial framework for classifying and interpreting those digital haptic emotions. To this end, we reviewed theoretical frameworks that have attempted to accomplish this goal with other nonverbal behaviors— most notably facial expressions and paralinguistics. Facial Expressions Research in facial expressions has received much attention from social scientists for the past fifty years. Some researchers argue that the face is a portal to one’s internal mental state (Ekman & Friesen 1978; Izard, 1971). These scholars argue that when an Virtual Interpersonal Touch 5 emotion occurs, a series of biological events follow that produce changes in a person—one of those manifestations is movement in facial muscles. Moreover, these changes in facial expressions are also correlated with other physiological changes such as heart rate or blood pressure (Ekman & Friesen, 1976). Alternatively, other researchers argue that the correspondence of facial expressions to actual emotion is not as high as many think. For example, Fridland (1994) believes that people use facial expressions as a tool to strategically elicit behaviors from others or to accomplish social goals in interaction. Similarly, other researchers argue that not all emotions have corresponding facial expressions (Cacioppo et al., 1997). Nonetheless, most scholars would agree that there is some value to examining facial expressions of another if one’s goal is to gain an understanding of that person’s current mental state. Ekman’s groundbreaking work on emotions has provided tools to begin forming dimensions on which to classify his set of six basic emotions (Ekman & Friesen, 1975). Figure 1 provides a framework for the facial classifications developed by those scholars.